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Abstract 


The  Software  Engineering  Institute  (SEIsm)  has  developed  two  methods  for  analyzing  system 
and  software  architectures — the  Quality  Attribute  Workshop  (QAW)  and  the  Architecture 

Tradeoff  Analysis  MethodSM  (ATAMsm).  These  techniques,  which  are  described  in  detail  in 
various  SEI  technical  reports  and  on  the  SEI  Web  site,  can  be  used  in  combination  to  obtain 
early  and  continuous  benefits.  Designed  to  complement  the  ATAM,  the  QAW  provides  a 
method  for  analyzing  a  conceptual  architecture  or  a  system  architecture  against  a  number  of 
critical  quality  attributes — such  as  availability,  performance,  security,  interoperability,  and 
modifiability — before  the  software  architecture  is  fully  developed.  Once  the  software  architec¬ 
ture  is  developed,  the  ATAM  can  be  used  to  reveal  how  well  the  architecture  satisfies  particu¬ 
lar  quality  attribute  requirements  and  the  risks,  sensitivities,  and  tradeoffs  involved  in 
satisfying  the  requirements. 

The  purpose  of  this  technical  note  is  to  describe,  using  a  hypothetical  example,  the  alignment, 
combination,  and  uses  of  the  two  methods. 
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1  Introduction 


Quality  Attribute  Workshops  (QAWs)  provide  a  method  for  analyzing  a  system’s  architecture 
against  a  number  of  critical  quality  attributes — such  as  availability,  performance,  security, 
interoperability,  and  modifiability — that  are  derived  from  mission  or  business  goals  [Barbacci 
02].  The  QAW  does  not  assume  the  existence  of  a  software  architecture.  It  was  developed  to 
complement  the  Architecture  Tradeoff  Analysis  MethodSM  (ATAMSM) '  in  response  to  cus¬ 
tomer  requests  for  a  method  that  identifies  important  quality  attributes  and  clarifies  system 
requirements  before  there  is  a  software  architecture  to  which  the  ATAM  could  be  applied2 
[Kazman  00],  The  QAW  can  be  applied  to  a  conceptual  (or  “notional”)  architecture  or  a  sys¬ 
tem  architecture.  The  QAW  and  the  ATAM,  which  are  described  in  detail  in  various  Software 
Engineering  Institute  (SEIsm)3  technical  reports,  in  the  book  Evaluating  Software  Architec¬ 
tures:  Methods  and  Case  Studies  [Clements  01  ],  and  on  the  SEI  Web  site  [SEI 02],  can  be  used 
in  combination  to  obtain  early  and  continuous  benefits.  It  should  be  noted  that  the  SEI  has 
developed  related  evaluation  techniques,  namely  the  Software  Architecture  Analysis  Method 
(SAAM,  a  predecessor  of  the  ATAM),  Active  Reviews  for  Intermediate  Designs  (ARID),  and 
the  Cost  Benefit  Analysis  Method  (CBAM),  which  are  not  covered  in  this  technical  note. 

In  an  ATAM  evaluation,  an  external  team  facilitates  meetings  between  stakeholders  during 
which  scenarios  representing  the  quality  attributes  of  the  system  are  developed,  prioritized, 
and  analyzed  against  the  architectural  approaches  chosen  for  the  system.  Typical  stakeholders 
include  developers,  users,  maintainers,  and  buyers.  The  results  of  the  analysis  are  expressed  as 
risks,  sensitivity  points,  and  tradeoffs.  To  conduct  an  ATAM  evaluation,  an  articulation  of  the 
business  drivers  and  an  initial  draft  of  the  software  architecture  are  required.  A  typical  ATAM 
evaluation  would  involve  2  two-to-three  day  meetings  between  the  evaluation  team  and  the 
stakeholders  over  the  course  of  a  few  weeks. 

The  QAW  involves  similar  activities  earlier  in  the  life  cycle  of  a  project.  In  the  QAW,  an  exter¬ 
nal  team  facilitates  meetings  between  stakeholders,  during  which  scenarios  representing  qual¬ 
ity  attribute  requirements  are  generated,  prioritized,  and  refined  (refining  involves  adding 
details  about  the  personnel  and  assets  required,  the  sequence  of  activities,  and  questions  about 


1  Architecture  Tradeoff  Analysis  Method  and  ATAM  are  service  marks  of  Carnegie  Mellon  University. 

2  The  ATAM  was  developed  to  evaluate  a  software  architecture  and  has  been  technically  validated  for  this  purpose.  Others 
have  applied  the  ATAM  to  other  types  of  architecture,  but  the  SEI  currently  makes  no  claims  about  the  ATAM’s  capabilities 
beyond  the  evaluation  of  a  software  architecture. 

3  SEI  is  a  service  mark  of  Carnegie  Mellon  University. 
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the  requirements).  After  the  scenario  generation  meeting(s),  the  refined  scenarios  are  con¬ 
verted  into  architectural  test  cases  that  the  architecture  team  analyzes  against  the  system  archi¬ 
tecture.  The  architectural  test  case  development  and  analysis  often  takes  place  over  an 
extended  period  of  time  (perhaps  months)  before  the  architecture  team  presents  the  results  of 
the  analysis  to  the  stakeholders. 

The  remainder  of  this  technical  note  describes  the  QAW  and  the  ATAM  and  uses  a  hypotheti¬ 
cal  example  (based  on  a  common  United  States  government  acquisition  strategy)  to  illustrate 
when  various  QAW  and  ATAM  activities  are  applicable. 
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2  QAW  Process 


The  QAW  process  shown  in  Figure  1  can  be  organized  into  four  distinct  groups  of  activities: 
(1)  scenario  generation,  prioritization,  and  refinement,  (2)  architectural  test  case  development, 
(3)  analysis  of  test  cases  against  the  system  architecture,  and  (4)  presentation  of  the  results. 

The  first  and  last  activities  of  the  process  occur  in  facilitated,  short  meetings  that  last  one  or 
two  days.  The.two  middle  activities  take  place  offline  and  can  continue  over  an  extended 
period  of  time.  Depending  on  the  application  context,  the  specific  roles  and  responsibilities  of 
the  participants  in  the  various  activities  can  be  customized  [Barbacci  02], 

The  process  is  iterative  in  that  the  architectural  test  case  analyses  might  lead  to  modifications 
of  the  architecture  that,  in  turn,  might  prompt  additional  test  case  analyses  that  result  in  a  con¬ 
tinuous  cycle  of  analyses  and  architectural  modifications.4 

This  document  describes  the  QAW  method  in  generic  terms.  The  actual  application  of  the 
method  can  be  tailored  to  the  needs  of  a  specific  organization  [Barbacci  02]. 


2.1  Scenario  Generation 

The  first  activity  in  the  QAW  process  is  to  generate,  prioritize,  and  refine  scenarios.  In  this 
process,  a  scenario  is  a  statement  about  some  anticipated  or  potential  use  or  behavior  of  the 
system;  it  captures  stakeholders’  concerns  about  how  the  system  will  do  its  job.  The  scenarios 
are  generated  during  a  facilitated  brainstorming  meeting  of  system  stakeholders  [Barbacci  02]. 

A  typical  agenda  for  this  meeting  is  shown  in  Table  1.  The  meeting  starts  with  a  facilitation 
team’s  presentation  of  an  overview  of  the  QAW  process,  including  QAW  activities  (the  ovals 
in  Figure  1)  and  their  expected  outcomes.  A  customer  representative  then  describes  the  sys¬ 
tem’s  mission  or  business  drivers,  including  the  business  context  for  the  system,  architectural 
drivers  (quality  attributes  that  “shape”  the  architecture),  and  critical  requirements  (quality 
attributes  that  are  most  central  to  the  system’s  success).  The  presentation  of  the  business  driv¬ 
ers  is  followed  by  an  overview  of  the  system  architecture.  The  overview  addresses  technical 
constraints  (such  as  an  operating  system,  hardware,  or  middleware  prescribed  for  use),  other 


Figure  1  does  not  include  planning  activities  such  as  planning  the  scenario  generation  meeting  because  these  activities 
might  require  a  number  of  interchanges. 
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Figure  1:  QAW  Process 
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Time 

Activity 

8:30  a.m. 

Start 

Welcome  and  introductions 

QAW  overview 

Business  drivers 

System  architecture 

10:00  a.m. 

Scenario  generation  and  prioritization 

Noon 

Lunch  break 

1:00  p.m. 

Scenario  refinement 

3:00  p.m. 

Wrap-up 

Review  scenarios 

Action  items 

4:00  p.m. 

End 

Table  1:  Agenda  for  QAW  Scenario  Generation 

systems  with  which  the  system  will  interact,  and  planned  architectural  approaches  to  address 
quality  attribute  requirements. 5  These  three  presentations  set  the  context  for  the  activities  that 
follow. 

Stakeholders  are  encouraged  to  generate  as  many  scenarios  as  possible  to  represent  a  wide 
range  of  concerns.  However,  only  a  small  number  of  scenarios  can  be  refined  during  a  one-day 
meeting.  Thus,  stakeholders  must  prioritize  the  scenarios  generated  previously  by  using  a  vot¬ 
ing  process;  they  must  then  refine  the  top  three  or  four  scenarios  to  provide  a  better  under¬ 
standing  of  them  in  terms  of  context  and  detail.  Prior  to  voting,  the  stakeholders  can  merge 
scenarios  that  they  consider  to  be  closely  related.  The  template  shown  in  Table  2  illustrates  the 
types  of  details  that  usually  emerge  from  the  refinement. 

The  result  of  this  meeting  is  a  prioritized  list  of  scenarios  and  refined  descriptions  of  the  top 
three  or  four  (merged)  scenarios  on  that  list. 


5  Depending  on  the  situation  (e.g.t  a  planned  competitive  acquisition  vs.  an  internal  corporate  development),  the  system  de¬ 
velopers  may  or  may  not  be  participants  in  this  meeting.  If  developers  are  excluded,  the  architecture  presentation  would  be 
made  by  a  customer  representative  and  would  describe  desired  rather  than  planned  approaches. 
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Section 

Content 

Reference  Scenario(s) 

The  scenario  that  is  being  refined  is  listed  here.  If  it  is  a  merged 
scenario,  the  combined  scenarios  are  listed  here  in  their  original 
form. 

Organizations 

Organizations  involved  in  or  affected  by  the  scenario(s) 

Actors/Participants 

Individuals  involved  in  or  affected  by  the  scenario(s) 

Quality  Attributes 

Quality  attributes  involved  in  or  affected  by  the  scenario(s) 

Context 

A  description  of  additional  details  such  as  the  environment  in 
which  the  scenario  takes  place,  and  the  sequence,  frequency,  and 
duration  of  events 

Questions 

Specific  questions  that  the  stakeholders  would  like  to  ask  the 
architect  and  the  designers  of  the  system.  Typically  one  or  more 
questions  are  included  for  each  quality  attribute  identified 
above,  for  example 

•  How  will  the  system  prevent  unauthorized  users  from 
accessing  the  system? 

•  How  will  the  system  store  one  year  of  information  online? 

•  How  will  the  system  respond  to  user  requests  within  10  sec¬ 
onds  during  peak  time? 

Table  2:  Template  for  QAW  Scenario  Refinement 


2.2  Architectural  Test  Case  Development 

The  objective  of  developing  architectural  test  cases  is  to  transform  each  refined  scenario  from 
a  statement  and  list  of  organizations,  participants,  quality  attributes,  and  questions  into  a  well- 
documented  architectural  test  case.  Test  cases  may  add  assumptions  and  clarifications  to  the 
context,  add  or  rephrase  questions,  group  questions  by  topic,  and  so  forth.  The  individual  or 
team  responsible  for  developing  test  cases  depends  on  the  situation.  Barbacci  and  associates 
describe  how  the  QAW  method  has  been  applied  and  who  carried  out  the  task  (such  as  the 
sponsor/acquirer  or  development  team)  [Barbacci  02]. 

An  architectural  test  case  has  a  context  section  that  outlines  the  important  aspects  of  the  case, 
an  issues  and  questions  section  that  states  the  stakeholders’  concerns,  and  a  graphical  illustra¬ 
tion  that  summarizes  these  issues  and  questions.  Figure  2  provides  a  visual  summary  of  the 
most  important  quality  attributes  and  the  specific  issues  and  questions  that  pertain  to  the 
attributes. 
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Quality 

Attribute 


Specific  Quality 
Attribute  Issue 


Question 


—  frequency  of  messages? 


performance 

...  time  to  send  a  message... 

...  latency  of  messages? 

...  time  to  perform  a  critical  function... 

...  priority  of  messages? 

...  question  I 

...  question  2 

...  time  to  restart  a  service... 

...  question  3 

...  question  1 

...  question  2 

...  question  3 

...  question  1 

attribute  1 

...  issue  1 

...  question  2 

...  issue  2 

...  question  3 

...  question  1 

...  question  2 

...  question  3 

attribute  2 


Figure  2:  Example  Illustration  of  Refined  Scenario  Issues  and  Questions 


2.3  Architectural  Test  Case  Analysis 

Typical  steps  for  conducting  the  architectural  test  case  analysis  include  the  following: 

1 .  Review  the  capabilities  of  the  assets  in  the  test  case  context  and  determine  how  the 
system  will  react  to  the  situation. 

2.  Make  and  document  any  assumptions  necessary  to  proceed  with  the  analysis. 

3.  Determine  which  architectural  views  (e.g.,  operational,  system,  technical,  process, 
behavioral,  structural)  can  best  describe  how  the  system  will  address  the  issues  and 
their  associated  questions. 
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4.  Perform  the  analysis  for  the  selected  architectural  test  cases. 

5.  If  necessary,  refine  the  architecture  to  help  answer  the  questions. 

6.  Document  the  answers  as  specifically  as  possible. 

The  analysis  for  a  specific  architectural  test  case  would  clarify  or  confirm  specific  quality 
attribute  requirements  and  might  identify  concerns  that  would  drive  the  development  of  the 
software  architecture.  Some  of  the  test  cases  could  later  be  used  as  “seed  scenarios”  in  an 
ATAM  evaluation  (e.g.,  to  check  if  a  concern  identified  during  the  test  case  analysis  was 
addressed  by  the  software  architecture).  The  results  of  analyzing  a  test  case  should  be  docu¬ 
mented  with  specific  architectural  decisions,  quality  attribute  requirements,  and  rationales. 


2.4  Presentation  of  Results 

The  presentation  is  a  one-  or  two-day  facilitated  meeting  attended  by  the  architecture  team  and 
other  stakeholders.  As  the  final  activity  in  the  QAW  process,  it  provides  an  opportunity  for  the 
architecture  team  to  present  the  results  of  their  analysis  and  demonstrate  that  the  proposed 
architecture  is  able  to  handle  the  architectural  test  cases  correctly. 

Prior  to  the  meeting,  participants  are  provided  with  a  short  document  describing  the  QAW  pro¬ 
cess,  business  drivers,  and  architectural  plans  that  were  presented  during  the  scenario  genera¬ 
tion  meeting.  In  addition,  the  document  includes  the  original  scenarios,  the  architectural  test 
cases,  and  an  example  of  a  test  case  analysis.  Ideally,  the  participants  would  be  the  same  stake¬ 
holders  who  took  part  in  the  scenario  generation  meeting.  However,  since  the  presentation  of 
results  might  take  place  a  few  months  after  the  first  meeting,  there  will  likely  be  some  partici¬ 
pants  who  were  not  involved  in  the  early  meeting.  The  short  document  serves  as  a  reminder  to 
those  participants  who  were  involved  in  the  scenario  generation  meeting  and  as  an  introduc¬ 
tion  to  the  QAW  process  for  new  participants. 

The  conclusions,  recommendations,  and  action  items  resulting  from  the  presentation  must  be 
captured  in  a  short  report  to  be  distributed  to  the  participants.  The  results  might  lead  to  modifi¬ 
cations  of  the  architecture,  which,  in  turn,  might  lead  to  further  analysis  of  test  cases  or  even 
new  test  cases.  These  iterations  are  shown  in  Figure  1. 
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3  ATAM  Process 


Since  the  ATAM  is  a  few  years  more  mature  than  the  QAW,  the  ATAM  process  is  more  pre¬ 
cisely  defined  and  has  been  more  rigorously  validated.  The  ATAM  process,  shown  in  Figure  3, 
is  organized  into  two  main  phases,  each  consisting  of  several  steps.  The  steps  in  each  phase 
usually  take  place  during  a  one-  to  two-day  facilitated  meeting,  while  the  two  phases  are  usu¬ 
ally  a  few  weeks  apart.6 

Phase  1  involves  a  small  group  of  predominantly  technically-oriented  stakeholders.  This  phase 
is  architecture-centric,  focused  on  eliciting  detailed  architectural  information  and  conducting  a 
top-down  analysis.  Phase  2  involves  a  larger  group  of  stakeholders.  This  phase  is  stakeholder¬ 
centric  and  focuses  on  eliciting  points  of  view  from  diverse  stakeholders  and  on  verifying  the 
results  of  Phase  1. 

The  ATAM  involves  nine  steps: 

1 .  Present  the  ATAM 

2.  Present  business  drivers 

3.  Present  architecture 

4.  Identify  architectural  approaches 

5.  Generate  quality  attribute  utility  tree 

6.  Analyze  architectural  approaches 

7.  Brainstorm  and  prioritize  scenarios 

8.  Analyze  architectural  approaches 

9.  Present  results 

The  first  three  steps  are  similar  to  the  QAW  presentations  made  before  scenario  generation  and 
the  QAW  presentation  of  results  meetings  in  that  they  inform  the  participants  about  the  pro¬ 
cess,  the  techniques  used,  and  the  expected  outcomes. 


6  The  complete  ATAM  process  includes  a  set  of  planning  steps  in  a  Phase  0.  These  steps  include  negotiations  and  planning 
tasks  that  might  take  a  number  of  interchanges  because  they  involve  scheduling  meetings,  selecting  facilities  and  partici¬ 
pants,  and  reviewing  documents.  Phase  0  can  extend  over  a  long  time  and  involves  multiple  interactions  with  the  customers. 
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Figure  3:  ATAM  Process 
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3.1  Phase  1 


In  Step  1,  the  evaluation  team  presents  an  overview  of  the  ATAM,  including  steps,  techniques, 
and  expected  outputs  (such  as  architectural  approaches,  a  utility  tree,  scenarios,  risks,  sensitiv¬ 
ity  points,  and  tradeoffs). 

In  Step  2,  a  customer  representative  describes  the  system’s  business  drivers,  including  the 
business  context  for  the  system,  high-level  functional  requirements,  high-level  quality 
attribute  requirements,  architectural  drivers  (quality  attributes  that  “shape”  the  architecture), 
and  critical  requirements  (quality  attributes  that  are  central  to  the  system’s  success). 

In  Step  3,  the  architect  presents  an  overview  of  the  architecture,  including  technical  constraints 
(such  as  an  operating  system,  hardware,  or  middleware  prescribed  for  use),  other  systems  with 
which  the  system  must  interact,  and  architectural  approaches  used  to  address  quality  attribute 
requirements. 

In  Step  4,  the  evaluators  begin  to  identify  places  in  the  architecture  that  are  key  to  realizing 
quality  attribute  goals;  they  also  identify  predominant  architectural  approaches  (some  exam¬ 
ples  include  client-server,  3-tier,  watchdog,  publish-subscribe,  and  redundant  hardware). 

In  Step  5,  the  participants  identify,  prioritize,  and  refine  the  most  important  quality  attribute 
goals  by  building  a  utility  tree.  A  utility  tree  is  a  top-down  vehicle  for  characterizing  the  “driv¬ 
ing”  attribute-specific  requirements.  The  most  important  quality  goals  are  the  high-level  nodes 
— typically  performance,  modifiability,  security,  and  availability.  Scenarios  are  the  leaves  of 
the  utility  tree,  as  illustrated  in  Figure  4. 

Scenarios  are  used  to  represent  stakeholders’  interests  and  should  cover  a  range  of  anticipated 
uses  of  the  system  (use  case  scenarios),  anticipated  changes  to  the  system  (growth  scenarios), 
or  unanticipated  stresses  to  the  system  (exploratory  scenarios).  A  good  scenario  clearly  indi¬ 
cates  which  stimulus  causes  it  and  what  responses  are  important.  During  scenario  prioritiza¬ 
tion,  scenarios  are  categorized  by  two  parameters,  importance  and  difficulty,  using  a  scale  of 
Low  (L)-Medium  (M)-High  (H). 

In  Step  6,  the  evaluation  team  probes  architectural  approaches  to  identify  risks,  sensitivity 
points,  and  tradeoffs  for  specific  quality  attributes.  A  risk  is  a  potentially  problematic  architec¬ 
tural  decision.  A  sensitivity  point  is  a  property  of  one  or  more  components  (and/or  component 
relationships)  that  is  critical  for  achieving  a  particular  quality  attribute  response.  A  tradeoff  is 
a  property  that  affects  and  is  a  sensitivity  point  for  more  than  one  attribute. 
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Figure  4:  Example  ATAM  Utility  Tree 


12 


CMU/SEI-2002-TN-005 


3.2  Phase  2 


At  the  beginning  of  the  Phase  2  meeting,  the  evaluation  team  and  some  of  the  participants 
from  the  Phase  1  meeting  briefly  recapitulate  the  results  of  the  Phase  1  meeting,  including  the 
quality  attributes  and  scenarios  in  the  utility  tree. 

In  Step  7,  the  stakeholders  generate  and  prioritize  scenarios  using  a  facilitated  brainstorming 
process.  These  scenarios  are  not  restricted  to  the  quality  attributes  listed  in  the  Phase  1  utility 
tree;  rather,  they  are  generated  to  represent  the  expanded  set  of  stakeholders’  interests.  How¬ 
ever,  the  scenarios  in  the  utility  tree  should  not  be  ignored;  these  scenarios  could  be  used  as 
example  scenarios. 

In  Step  8,  the  stakeholders  identify  the  architectural  approaches  affected  by  the  scenarios  gen¬ 
erated  in  Step  7.  Step  8  continues  the  analysis  started  in  Step  6  of  Phase  1  using  the  new  sce¬ 
narios  and  identifying  additional  risks,  sensitivity  points,  and  tradeoffs. 

Finally,  in  Step  9,  the  evaluation  team  presents  the  ATAM  outputs  to  the  stakeholders  as  con¬ 
firmation  of  the  architectural  approaches,  utility  tree,  scenarios,  risks,  sensitivity  points,  and 
tradeoffs  identified  during  the  exercise. 
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4  An  Example  Application  of  Both 
Methods 


There  are  often  contexts  in  which  both  the  QAW  and  the  ATAM  can  be  used  on  the  same  sys¬ 
tem.  Figure  5  illustrates  a  common  United  States  government  acquisition  strategy.  Starting 
with  an  initial  request  for  proposals  (RFP),  an  acquisition  organization  evaluates  proposals 
from  multiple  contractors  and  awards  contracts  to  a  small  number  of  contractors  to  conduct  a 
Competitive  Fly-Off.  At  the  end  of  the  Competitive  Fly-Off,  the  contractors  submit  updated 
technical  proposals,  including  additional  details,  and  the  acquirer  makes  a  Final  Down  Select. 
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Figure  5:  A  Common  Acquisition  Strategy  [Bergey  02] 


We  will  use  this  example  to  illustrate  how  QAW  and  ATAM  activities  can  be  combined.  In  this 
case,  both  methods  are  used  during  the  acquisition  process. 


4.1  Activities  During  the  Competitive  Solicitation 
Phase 

Figure  6  shows  how  QAW  activities  can  be  incorporated  during  the  Competitive  Solicitation 
Phase. 
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1 .  One  or  more  QAW  scenario  generation  meetings  are  conducted  with  teams  from  the 
acquiring  organization  (e.g.,  representative  groups  of  stakeholders  with  similar  needs 
and  responsibilities). 

2.  From  these  scenarios,  the  acquiring  organization  develops  a  collection  of  architectural 
test  cases  that  represent  concerns  of  the  stakeholders. 

3.  Early  in  the  Competitive  Solicitation  phase  and  prior  to  the  release  of  the  RFP,  the 
acquirer  conducts  bidders’  conferences  to  inform  potential  bidders  about  the  need  for 
conducting  architectural  analysis. 

4.  The  acquirer  drafts  sections  of  the  RFP  to  incorporate  architectural  analysis  require¬ 
ments  and  includes  the  architectural  test  cases  as  government-furnished  items  in  the 
RFP  proper. 

5.  As  part  of  their  proposals,  bidders  are  expected  to  describe  how  they  will  conduct  the 
architectural  analysis  [Bergey  02], 
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Figure  6:  QAW  Activities  During  the  Competitive  Solicitation  Phase 
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4.2  Activities  During  the  Competitive  Fly-Off  Phase 

Figure  7  shows  how  QAW  activities  can  be  incorporated  during  the  Competitive  Fly-Off 
phase. 

1.  The  contractors  analyze  their  architectures  against  the  architectural  test  cases  and,  if 
necessary,  modify  their  planned  architecture. 

2.  The  contractors  present  the  results  of  their  analysis  in  both  a  dry-run  (rehearsal)  as 
well  as  a  final  presentation. 

A  dry-run  presentation  should  be  conducted  when  the  architecture  team  making  the 
presentation  is  unsure  about  any  of  the  following: 

•  the  level  of  detail  required 

•  the  precision  expected  from  its  answers  to  the  architectural  test  case  questions 

•  how  to  incorporate  other  analysis  results  (such  as  reliability,  availability,  and 
maintainability  analysis  or  network-loading  analysis) 

•  what  additional  architectural  documents  might  be  needed 

The  final  presentation  takes  place  after  the  contractors  polish  the  results  of  the  dry-run 
presentation. 

3.  After  resolving  any  potential  concerns  resulting  from  the  presentation  of  results,  the 
contractors  present  their  technical  proposals  to  the  acquiring  organization. 
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Figure  7;  QAW  Activities  During  the  Competitive  Fly-Off  Phase 
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4.3  Activities  During  the  System  Implementation 
Phase 

Figure  8  shows  how  ATAM  activities  can  be  incorporated  during  the  System  Implementation 
phase. 

1 .  The  preceding  QAW  activities  have  already  identified  the  important  quality  attribute 
concerns  that  can  be  used  to  draft  the  beginning  of  the  utility  tree  (Step  5  in  Phase  1  of 
the  ATAM). 

The  draft  utility  tree  could  be  further  augmented  with  “seed  scenarios”  derived  from 
selected  architectural  test  cases,  to  determine  whether  concerns  identified  during  the 
test  case  analyses  were  addressed  by  the  software  architecture. 

2.  ATAM  evaluations  could  be  scheduled  to  fit  the  development  plans,  although  the 
results  of  previous  QAW  architectural  test  case  analyses  might  influence  the  schedule. 
For  example,  components  or  subsystems  that  were  identified  as  sources  of  concern  to 
the  developers  might  be  subject  to  ATAM  evaluations  earlier  than  other  subsystems. 
These  subsystems  could  also  be  subject  to  multiple  ATAM  evaluations  during  devel¬ 
opment. 

3.  Risks  identified  during  the  ATAM  evaluations  should  be  addressed  during  system 
development. 

Depending  on  the  circumstances  and  the  results  of  the  QAW  architectural  test  case  analyses, 
ATAM  phases  could  be  customized  to  allow,  for  example,  one  Phase- 1  evaluation  (involving 
mostly  developers)  and  multiple  Phase-2  evaluations  (involving  different  groups  of  stakehold¬ 
ers,  such  as  clients  or  users  with  different  needs). 
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Figure  8:  ATAM  Activities  During  the  System  Implementation  Phase 
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5  Summary  Comparison 


Table  3  provides  a  brief  summary'  of  the  inputs,  outputs,  and  participants  involved  in  the  vari¬ 
ous  QAW  activities  and  ATAM  Phases. 


Inputs 

Outputs 

Participants 

Duration 

QAW 

Scenario 

Generation 

QAW  presentations 
(business  drivers,  con¬ 
ceptual  or  system  archi¬ 
tecture,  quality  attribute 
requirements) 

prioritized, 
refined  scenar¬ 
ios 

system  stakeholders 
(see  presentation  of 
results). 

Depending  on  the 
situation,  they  may 
or  may  not  include 
system  developers. 

one- or  two-day 
facilitated  meet¬ 
ings 

QAW 

Architectural 
Test  Case 
Development 

prioritized,  refined  sce¬ 
narios 

architectural 

test  cases 

Depending  on  the 
situation,  the  partici¬ 
pants  might  be  sys¬ 
tem  developers  or 
acquirers/sponsors. 

, , .  could  take 
several  days 

QAW 

Architectural 
Test  Case 
Analysis 

architectural  test  cases 

results  of  archi¬ 
tectural  test  case 
analysis 

system  developers 

. . .  could  take 
weeks  or 
months 

QAW 

Presentation 
of  Results 

results  of  architectural 
test  case  analysis 

additional 
results  and  anal¬ 
ysis  report 

presentation  made 
by  system  develop¬ 
ers  to  other  system 
stakeholders  (see 
scenario  generation) 

one-  or  two-day 
facilitated  meet¬ 
ings 

ATAM 

Phase  1 

ATAM  Presentations 
(business  drivers  and 
architectural  approaches) 

utility  tree, 
risks,  sensitivi¬ 
ties,  and 
tradeoffs 

sponsors  and  devel¬ 
opers  (technically 
oriented  stakehold¬ 
ers) 

one-  or  two-day 
facilitated  meet¬ 
ings 

ATAM 

Phase  2 

ATAM  Presentations 
(business  drivers  and 
architectural 
approaches); 
results  from  Phase  1 
(utility  tree,  risks,  sensi¬ 
tivities,  and  tradeoffs) 

results  from 
Phase  1 ,  addi¬ 
tional  scenar¬ 
ios,  risks, 
sensitivities, 
and  tradeoffs 

sponsors,  develop¬ 
ers,  and  other  stake¬ 
holders  (such  as 
customers,  users, 
maintained) 

one-  or  two-day 
facilitated  meet¬ 
ings 

Table  3:  QAW  and  ATAM  Inputs,  Outputs,  and  Participants  for  Various  Activities 
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